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Abstract. Serious games can help people function more effectively in complex settings, facilitate 
their role as team members, and provide insight into their team's mission. In such games, 
coordination and cooperation among team members are foundational to the mission's success and 
provide a preview of what individuals and the team as a whole could choose to do in a real scenario. 
Serious games often model events requiring life-or-death choices, such as civilian rescue during 
chemical warfare. How the players communicate and what actions they take can determine the 
number of lives lost or saved. However, merely playing a game is not enough to realize its most 
practical value, which is in learning what actions and communication methods are closest to what the 
mission requires. Teams often play serious games in isolation, so when the game is complete, an 
analytical stage is needed to extract the strategies used and examine each strategy’s success 
relative to the others chosen. Recognizing the importance of this next stage, Noblis has been 
developing Game Analysis, software that parses individual game play into meaningful units and 
generates a strategic analysis. Trainers create a custom game-specific grammar that reflects the 
objects and range of actions allowable in a particular game, which Game Analysis then uses to parse 
the data and generate a practical analysis. Trainers have then enough information to represent 
strategies in tools, such as Gantt and heat map charts. First-responder trainees in North Carolina 
have already partnered Hot-Zone and Game Analysis with great success. 

analysis. Game Analysis uses the Extensible 
Markup Language (XML) to import every 
game-play action and communication. 
Trainers create a custom game-specific 
grammar (ontology) that reflects the objects 
and range of actions allowable in a particular 
game, which Game Analysis then uses to 
parse the data and generate a practical 
analysis. Trainers have enough information to 
represent strategies in Gantt and heat map 
charts, for example. 

First-responder trainees in North Carolina 
have already partnered Hot-Zone and Game 
Analysis with great success. Trainees found 
the game easy to play and trainers deemed 
Game Analysis valuable in defining what 
strategies worked best. Game Analysis also 
helped identify individuals who were 
particularly well-suited for certain aspects of 
their functional responsibilities. Trainers could 
see “at a glance” who was the best at 
communicating status or performing triage, for 
example. This larger view changed the 
training exercise from a local trainee 
perception to the bigger picture of the entire 
mission. 

2. HOT-ZONE GAME PLAY 

In Hot-Zone, players act as either hazardous 
materials (hazmat) technicians or the incident 


1. INTRODUCTION 

Multiplayer, interactive serious games can 
help people function more effectively in 
complex settings, facilitate their role as team 
members, and provide insight into their team’s 
mission. In such games, coordination and 
cooperation among team members are 
foundational to the mission’s success and 
provide a preview of what individuals and the 
team as a whole could choose to do in a real 
scenario. Serious games often model events 
requiring life-or-death choices, such as civilian 
rescue during chemical warfare. How the 
players communicate and what actions they 
take can determine the number of lives lost or 
saved. 

However, merely playing a game is not 
enough to realize its most practical value, 
which is in learning what actions and 
communication methods are closest to what 
the mission requires. Teams often play 
serious games in isolation, so when the game 
is complete, an analytical stage is needed to 
extract the strategies used and examine each 
strategy’s success relative to the others 
chosen. 

Recognizing the importance of this next stage, 
Noblis has been developing Game Analysis, 
software that parses individual game play into 
meaningful units and generates a strategic 
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commander in responding to the release of 
chlorine gas in a shopping mall. With only 
limited equipment, they must decide how they 
will rescue and decontaminate as many 
civilians as possible in the shortest time. Their 
specific tasks are to neutralize the gas source; 
set up a security perimeter and 
decontamination tent; triage, tag, and 
evacuate people; hand over evacuated people 
for decontamination and decontaminate 
themselves. 




(b) 


Figure 1; Playing Hot-Zone. During game 
play, the user must choose a strategy for 
evacuating and treating victims from a 
chemical gas explosion at a mall, (a) The 
interface lets the scenario manager tailor the 
overall difficulty of the game scenario by 
adjusting parameters such as the number of 
victims, chemical gas potency, and the 
decontamination tent efficiency. Such tailoring 
also allows for various skill levels, (b) The 
user’s avatar is a hazardous materials 
technician evacuating a victim. Bars (left 
foreground) display the user’s degree of 
exhaustion (cross) and pace (running man 
icon), while a smart-strip-like indicator (right) 
shows the degree of gas potency. 

Figure 1 shows screen shots from Hot-Zone, 
which typically takes between 15 to 30 
minutes to play, depending on the difficulty 
level the trainer chooses for that responder 
trainee. To accommodate variations in 
particular scenarios, trainers can adjust 


parameters such as the number of 
participants and victims, amount of 
equipment, first responder’s skill level; and 
effectiveness of the measures taken. With this 
customization flexibility, trainers can conduct 
what-if exercises, such as examining 
anticipated situations or ideal responses to 
hypothetical events. 

3. ANALYZING HOT-ZONE DATA 

Given that Hot-Zone’s main goal is to improve 
team performance, trainers need enough 
information to correct team behavior, which 
requires more insight than just numbers of 
lives saved and lost. Many analysis programs 
and game engines allow only a high level data 
capture, which trainers then use as a restart 
point, essentially a second chance for the 
player to face a game situation and make 
different choices. Such programs rarely 
provide enough insight for trainers to alter 
team behavior. 

Thus, for Game Analysis, a key goal was to 
enable the capture and analysis of all the data 
generated during game play. In this way, 
trainers could see the entire range of a 
player’s actions and communications, not just 
select blocks. Game Analysis stores these 
actions and communications in a database, 
along with relevant metadata such as 
beginning and end times, event triggers, and 
who is sending and receiving the messages. 

As Figure 2 part 1 shows, the game-specific 
grammar expresses the game’s actions and 
events at an atomic level and groups these 
atoms into composite actions and 
happenings, which Game Analysis uses to 
discover game events. By filtering the 
composites, it can then obtain player, team, or 
environmental views and can parse an event 
sequence into a hierarchical task-oriented 
description of strategy, as in Figure 2 part 2. 

Figure 2 part 3 shows how Game Analysis 
can aggregate individual strategies to produce 
an overall team strategy. The game-specific 
grammar naturally expresses any task 
hierarchy, and because each task has a start 
and end time (time-stamping), Game Analysis 
can capture the task hierarchy and express it 
in analysis tools such as the Gantt chart in 
Figure 2 part 4. 

Obviously, creating a game-specific grammar 
requires a detailed analysis of the particular 
game, which at present an expert must do, 
but at some point a software novice might be 
able to do through a user interface. The long 
term vision is to provide enough flexibility for 
trainers to use commercial off-the-shelf 
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(COTS) software to assemble a custom 
training program. 

Another benefit of Game Analysis is that it 
captures the performance and style of play for 
individual trainees, which trainers can then 
relate to that trainee’s professional 
background, skills, and training. Such factors 
have direct bearing on strategy development. 
Changing the skill mix and the way players 
communicate among themselves and with 
their incident commander can greatly 


influence the outcome of scenarios in game 
play. For example, trainees frequently delayed 
reporting to the incident commander that 
decontamination showers were ready. This 
communication delay often meant a longer 
total time to complete the evacuation 
successfully. Such decisions are often 
overlooked in games that rely almost 
exclusively on data about victims saved or 
lost. 
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Figure 2: Using a game-specific grammar to parse atomic events into subtasks. (1) The row of 
colored squares represents atomic events that make sense in a particular game essentially all 
actions taken. (2) Applying the game-specific grammar, Game Analysis can sort these actions into 
specific subtasks. For example, in Hot-Zone, Subtask 2 could be defined as consisting of the 
sequence pick up victim (orange square), carry victim (purple square), and drop victim (blue square). 
(3) Game Analysis provides pattern recognition by breaking subtasks into their start and end times 
and can then generate useful analysis tools, such as (4) a Gantt chart to show the tasks performed 
and how long it took to perform them. 


4. APPLICATION RESULTS 

In July 2007, Noblis tested the partnership of 
Hot-Zone and Game Analysis on trainees at 
Wake Technical Community College (WTCC) 
in Raleigh, North Carolina, which trains more 
than a thousand law enforcement and first 
responders yearly. The tests took place for 
one week at 15 fire stations in the Raleigh 
area, typically two play sessions daily. The 
first responders did not operate Game 
Analysis but were shown the analysis results 
at the end of the second game. Overall 25 
different teams played the game. 

Overall, first responders were enthusiastic 
about playing Hot-Zone. In about 15 minutes, 
all players in all departments — no matter how 


old or young or even injured, which is 
common among first responders — could 
master game play using the keyboard and 
mouse controls, in part because even the 
older players had some experience with 
gaming for entertainment. Only one team 
member had difficulty, which he attributed to 
limited computing experience. Players were 
very professional during game play, and their 
professionalism intensified as the scenarios 
became more challenging. Figure 3 shows a 
typical level of concentration and interest. At 
the end of the game, each player saw a quick 
team performance summary, and a week 
later, many viewed the results of analyzing 
their games with Game Analysis (some 
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players had corrupt data sets that could not 

be analyzed). 



Figure 3: First responder trainees from Wake 
Technical Community College intently playing 
Hot-Zone. WTCC trains more than a thousand 
law enforcement and first responders yearly. 

Each of the 25 teams had the same four 
scenarios in the same order. Beginner: 
introduces teams to game controls and 
communication mechanisms. Easy: mall 

scenario with only a few people and so only a 
few victims need rescuing. Medium: same as 
Easy but with a population more 


representative of an actual mall. Hard: Same 
setting as Medium but with a secondary 
explosion and a heavy dose of chlorine. 

Figure 4 shows the results of two WTCC 
teams. Both teams were experienced first 
responders, yet Team A had five fatalities and 
Team B had none. In addition, Team A took 
33 percent longer than Team B to complete all 
the essential tasks (40 minutes vs. 30 
minutes). 

The charts in the figure reveal some 
interesting insights into why Team B’s 
performance was superior. Team A took the 
seemingly natural step of starting to evacuate 
as soon as possible. In contrast, Team B’s 
initial focus was on finding and neutralizing 
the source of the chlorine gas. Team A also 
deferred decontaminating team members until 
the end, while Team B opted to perform at 
least some team decontamination in parallel 
with decontaminating the evacuees. 

These differences have important implications 
for strategy. The more successful strategy 
was to recognize and attend to the most 
significant obstacle to overall success — in this 
case, the source of the chlorine gas — rather 
than to rush immediately to improve a score, 
such as evacuating victims right away in the 
mistaken notion that this will save more lives. 
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Figure 4: Gantt charts from Hot-Zone game play. Given the same initial situation, Teams A and B 
chose different strategies, leading to different outcomes. Examining game outcomes in terms of 
choices made leads to better performance, since the focus is on mission success, not on a particular 
score. 
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5. TOWARD A GAMING PLATFORM 

Numerous commercially available systems 
engineering tools provide disciplined 
approaches to examining user and system 
requirements and operational and system 
performance. These systems engineering 
tools can combine with a serious game and 
Game Analysis to form an integrated 
framework. A framework, such as that in 
Figure 5, makes it possible to develop and 
test operational concepts and system designs. 

The serious game play stage of the 
framework is the game environment, which 


consists of the game to be played, a game 
scenario editor that the trainer operates, and a 
quick reporting service that displays limited 
results after each game play session. The 
reporting service generates XML logs that can 
serve as input to Game Analysis. The pattern 
recognition algorithm in Game Analysis then 
analyzes the raw data and generates a 
strategy hierarchy, as described earlier. The 
strategy hierarchy becomes the basis for 
generating visualizations of the captured 
strategy and an XML representation of the 
strategy that is compatible with the chosen 
systems engineering tool. 


Serious Game Play Game Analysis COTS Systems Engineering Tool 



Figure 5: An integrated framework for playing serious games and analyzing results. The framework 
consists of the game play environment, Game Analysis for characterizing strategies, and any XML- 
compatible system engineering tool. The best strategies identified in Game Analysis become the 
basis for recommended processes, which are straightforward to represent in systems engineering 
tools. 


Noblis has already used the framework with 
Hot-Zone and Mega International’s Mega 
modeling suite systems engineering tool 
(www.mega.com). Using Mega Designer, 
Noblis analysts generated a swim-lane 
representation of how the team operated. In 
this representation, each first responder has 
his own lane” with the time axis going from 
the top of the page to the bottom. Individual 
lanes show the actions that a particular player 
takes and the time they were taken. The 
swim-lane representation also shows the 
communications among players as well as the 
actions that several players took jointly. 

When supplemented with strategy extraction 
and an analysis capability, serious games are 
an effective way to improve operational 
performance, providing feedback on 
strategies chosen and empirically finding what 
works best. 

The use of serious games for training 
individuals and teams will continue to grow, as 
more organizations realize their value in 
letting trainees experiment with different 


strategies and visualize their results. By 
working through a variety of scenarios, 
players begin to develop a feel for what will 
work and for how to act more strategically in 
emergency situations. Ultimately, they will 
begin to view a mission more systematically 
and adapt strategies that lead to the overall 
successful resolution of an emergency. 

A more advanced and practical use of the 
game analysis framework is in the design of a 
dynamic and adaptive operation management 
tool. By collecting and analyzing hundreds of 
plays and scenarios from many first responder 
teams, it might be possible to generate an 
artificial-intelligence-like algorithm that could 
provide recommendations on resource 
allocations and task priority to an incident 
commander. The incident commander could 
then use the tool by plugging in the current 
state of an ongoing operation and receive 
actionable recommendations to optimize its 
current strategy. This application could lead to 
more immediate correct actions in a crisis. 
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